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Abstract 

'^Distributed  artificial  intelligence  is  the  study  of  how  a  group  of  individual  intelligent 
agents  can  combine  to  solve  a  difficult  global  problem;  the  usual  approach  is  to  split 
the  original  problem  into  simpler  ones  and  to  attack  each  of  these  independently.  This 
paper  discusses  in  very  general  terms  the  problems  which  arise  if  the  subproblems  are  not 
independent,  but  instead  interrelate  in  some  way.  We  arc  led  to  a  single  assumption,  which 
we  call  common  rationality,  that  is  provably  optimal  (in  a  formal  sense)  and  which  enables 
us  to  characterize  precisely  the  communication  needs  of  the  participants  in  multi-agent 
interactions.  An  example  of  a  distributed  computation  using  these  ideas  is  presented. 


§1.  Introduction 


The  thrust  of  research  in  distributed  artificial  intelligence  (DAI)  is  the  investigation 
of  the  possibility  of  solving  a  difficult  problem  by  presenting  each  of  a  variety  of  machines 
with  simpler  parts  of  it. 

The  approach  that  has  been  taken  has  been  to  consider  the  problem  of  dividing  the 
original  problem:  what  subtasks  should  be  pursued  at  any  given  time?  To  which  avail¬ 
able  machine  should  a  given  subtask  be  assigned?  The  question  of  how  the  individual 
machines  should  go  about  solving  their  subproblcms  has  been  left  to  the  11011-distributcd 
AI  community  (or  perhaps  to  a  recursive  application  of  DAI  techniques). 

The  assumption  underlying  this  approach  -that  each  of  the  agents  involved  in  the 
solution  of  the  subproblcms  can  proceed  independently  of  the  others— has  recently  been 
called  into  question  [2,3,6,7,10].  It  has  been  realized  that,  in  a  world  of  limited  resources, 
it  is  inappropriate  to  dedicate  a  substantial  fraction  of  those  resources  to  each  processor. 
The  increasing  attractiveness  of  parallel  architectures  in  which  processors  share  memory 
is  an  example  of  this:  memory  is  a  scarce  resource. 

Automated  factories  must  inevitably  encounter  similar  difficulties.  Arc  the  robots 
working  in  such  factories  to  be  given  distinct  bins  of  component  parts,  and  non-overlapping 
regions  in  which  to  work  or  to  travel  from  one  area  of  the  factory  to  .another?  It  seems 
unlikely. 

My  intention  in  this  paper  is  to  discuss  these  issues  at  a  very  basic  (i.e.,  formal)  level. 
I  will  be  interested  in  situations  where: 

(1)  A  global  goal  has  been  replaced  by  a  variety  of  local  ones,  each  pursued  by  an  indi¬ 
vidual  agent  or  process,  and 

(2)  The  actions  of  the  individual  agents  may  interact,  in  that  success  or  failure  for  one 
such  agent  may  be  partially  or  wholly  contingent  upon  an  action  taken  by  another. 
The  first  of  these  is  a  massive  disclaimer.  1  will  not  be  concerned  with  the  usual 

problem  of  dividing  the  global  goal,  but  will  assume  that  this  has  already  been  done.  My 
intention  is  merely  to  remove  the  constraint  that  the  subproblems  or  subagents  cannot 
interact;  the  problem  of  subdividing  problems  in  the  absence  of  this  constraint  is  outside 
the  intended  scope  of  this  paper. 

The  second  remark  above  explicitly  .allows  for  “partial”  success  or  failure  -  correspon¬ 
ding,  for  example,  to  the  speed  with  which  a  given  subgoal  is  achieved. 


In  the  case  where  the  agents  do  not  interact,  we  will  assume  that  each  knows  enough  to 
evaluate  the  results  of  its  possible  actions.  For  agent  i,  this  evaluation  will  be  incorporated 
in  a  payoff  function  p  which  assigns  to  any  alternative  m  the  value  of  that  course  of  action. 
Thus  if  M  represents  the  set  of  all  of  t’s  possible  courses  of  action,  p  is  simply  a  function 

p  :  M  -♦  1R.  (1) 

That  the  range  of  p  is  H  as  opposed  to  {0, 1}  reflects  our  allowing  for  a  varying  degree  of 
success  or  failure. 

Determining  the  function  p  is  a  task  that  lies  squarely  within  the  province  of  non- 
distributed  AI  research.  Given  such  a  function,  selecting  the  alternative  m  which  maxi¬ 
mizes  p(m)  is  straightforward. 

In  the  absence  of  interaction,  the  problems  of  the  individual  agents  would  now  be 
solved.  If  the  success  or  failure  of  some  agent  depends  on  actions  taken  by  another, 
however,  this  is  not  the  case:  the  function  p  in  (l)  will  have  as  its  domain  the  set  of 
actions  available  to  all  of  the  agents,  as  opposed  to  a  single  one. 

This  sort  of  problem  has  been  discussed  extensively  by  game  theorists.  They  do, 
however,  generally  make  the  assumption  that  the  agents  have  common  knowledge  of  each 
others’  payoff  functions.  This  need  not  be  valid  in  an  AI  setting. 

We  will  address  this  problem  in  due  course;  our  basic  view  is  that  the  fundamental  role 
of  communication  between  interacting  intelligent  agents  is  to  establish  an  agreed  payoff 
function  as  described  in  the  last  paragraph.  Before  turning  to  this,  let  us  examine  in  an 
informal  way  some  situations  in  which  interaction  is  important. 

The  first  is  one  which  we  will  refer  to  as  coordination.  Suppose  that  two  robots,  one  in 
Boston  and  the  other  in  Palo  Alto,  decide  to  meet  in  Pittsburgh  to  build  a  widget.  Now, 
building  a  widget  is  a  complicated  procedure  unless  you  happen  to  have  both  a  zazzyfrax 
and  a  borogovc,  although  cither  in  isolation  is  useless.  Zazzyfraxen  arc  available  only  in 
New  York,  and  borogoves  only  in  San  Francisco;  should  the  robots  stop  on  their  travels  to 
acquire  their  respective  components? 

The  answer  is  clearly  that  they  should;  note,  however,  that  for  cither  robot  to  do  so 
involves  a  great  many  assumptions  about  the  other  robot.  The  Palo  Alto  robot  needs  not 
only  to  know  about  the  availability  of  zazzyfraxen  in  New  York — he  must  also  assume  that 
the  Boston  robot  knows  about  the  borogoves  in  San  Francisco,  and  that  the  Boston  robot 
knows  the  Palo  Alto  robot  knows  about  the  zazzyfraxen,  and  so  on.  Halpern  and  Moses 
[8]  have  described  this  sort  of  situation  os  common  knowledge. 

As  we  remarked  earlier,  common  knowledge  of  this  sort  is  presumed  by  the  game 
theorists  in  their  assumption  of  common  knowledge  of  the  payoff  function.  But  even  this 
is  not  enough  to  ensure  coordination  of  the  robots’  actions:  the  Palo  Alto  robot  must 
assume  that  the  Boston  robot  is  sensible  enough  to  stop  in  New  York,  which  implies  that 
the  Boston  robot  knows  he  is  sensible  enough  to  stop  in  San  Francisco,  and  so  on.  So  even 
common  knowledge  of  the  payoff  function  is  not  enough:  some  sort  of  common  knowledge 
of  the  methods  by  which  the  robots  select  their  actions  is  also  required. 

The  game  theorists  deal  with  this  last  requirement  by  fiat,  assuming  [13]  that  any 
“rational”  agents  will  coordinate  their  actions  in  such  a  situation.  This  seems  unnecessarily 
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ad  hoc ,  and  also  appears  to  conflict  with  the  philosophy  that  each  agent  strives  to  achieve 
a  purely  local  goal. 

Our  robots  somehow  overcome  these  difficulties,  starting  a  successful  widget  business 
in  Pittsburgh.  At  some  point,  however,  their  zazzyfrax  breaks,  and  they  arrange  with 
the  New  York  factory  to  exchange  five  widgets  for  a  replacement.  Since  the  robots  need 
the  zazzyfrax  as  quickly  as  possible  and  the  New  York  factory  is  equally  anxious  to  get 
its  widgets,  it  is  decided  that  each  group  will  ship  their  contribution  to  the  exchange 
immediately,  trusting  the  other  to  act  similarly. 

In  the  absence  of  legal  sanctions,  why  should  either  agent  ship  the  goods?  The  result 
of  this,  however,  is  for  there  to  be  no  transaction  -an  outcome  worse  for  both  agents  than 
if  they  had  acted  as  agreed. 

The  problem  is  one  of  cooperation ;  the  scenario  we  have  presented  is  Hofstadter’s  [9] 
reformulation  of  the  well-known  prisoner’s  dilemma. 

The  game  theorists  have  no  answer  in  this  case.  Not  only  does  the  pursuit  of  a  local 
goal  offer  no  incentive  for  cooperation,  it  actively  discourages  it.  The  robots’  business, 
having  become  too  dependent  on  the  zazzyfrax  technology,  goes  bankrupt. 

§2.  Solutions  to  problems  of  interaction 

Our  example  may  have  been  far-fetched,  but  the  problems  are  real.  The  coordination 
problem  has  straightforward  analogs  in  automated  factories,  as  docs  the  cooperation  one: 
why  should  one  robot  or  .agent  stop  to  arrange  delivery  of  a  part  required  by  another  when 
doing  so  docs  not  further  its  own  local  ambitions? 

A  variety  of  solutions  has  been  proposed.  The  one  which  has  received  the  most 
attention  in  the  A1  literature  (2,3,12]  has  been  to  ensure  that  the  agent  responsible  for 
dividing  the  original  problem  into  more  manageable  smaller  ones  does  so  in  a  way  which 
ensures  that  the  subtasks  do  not  interact.  We  have  already  remarked  that  this  may  be 
overly  constraining  in  many  situations. 

It  has  also  been  suggested  that  the  agents  communicate  their  intentions  to  one  another. 
Roscnschciu  [15]  has  demonstrated,  however,  that  there  is  no  reason  for  independent  agents 
to  communicate  honestly.  Since  a  statement  of  the  form,  “1  intend  to  do  x,”  is  virtually 
improvable,  the  value  of  such  communication  is  extremely  limited.  Indeed,  it  was  the  non¬ 
binding  nature  of  communication  that  led  to  the  downfall  of  the  cntrcprenucrial  robots 
of  the  last  section.  Roscnschein  and  Gcncsercth  [14]  have  suggested  remedying  this  by 
making  the  commitments  binding,  but  it  is  not  clear  how  this  can  be  enforced. 

The  game  theorists  use  different  approaches.  Wc  have  already  remarked  upon  their 
adoption  of  a  definition  of  rationality  that  considers  coordinated  action;  it  unfortunately 
does  not  seem  possible  to  formalize  this  in  a  way  that  is  clearly  consistent  with  the  “local 
utility  maximization”  that  is  generally  assumed.  The  game  theorists  themselves  have 
remarked  upon  this  [11]. 

Cooperation  is  harder  to  come  by  than  is  coordination.  The  game  theorists  achieve 
it  by  assuming  there  is  some  facility  for  retribution;  this  amounts  to  assuming  that  the 
given  interaction  is  only  one  of  a  scries.  Axelrod  [l]  has  shown  that  cooperation  can  evolve 
under  these  assumptions. 

Again,  the  view  that  an  agent’s  actions  should  be  governed  by  a  concern  for  its  future 
welfare  is  not  in  keeping  with  the  idea  that  such  an  agent  should  act  to  maximize  a  purely 


local  payoff.  Wc  have  similar  objections  to  schemes  that  include  au  ad  hoc  “altruism 
factor”  in  the  payoff  function. 


§3.  Notation 

In  order  to  formalize  a  solution  to  these  difficulties,  we  need  first  to  formalize  the 
problem.  Wc  begin  by  generalizing  (1).  There,  for  each  agent  i,  wc  had  a  function 

Pi  :  Mi  — •>  It, 

where  Af,  is  the  set  of  allowable  “moves”  for  player  i.  In  the  interactive  case,  this  generalizes 
to  a  function 

P.  :  JJ  Mi  — >  It,  (2) 

i€P 

where  P  is  the  set  of  all  players  involved  in  the  interaction. 

For  S  C  P,  wc  will  denote  P  —  S  by  S]  we  will  also  write  simply  i  instead  of  {t}  where 
no  confusion  is  possible.  (Thus  i  =  P  —  {i},  for  example.)  Wc  also  write  Ms  for  Hiev  ^i, 
so  that  (2)  becomes 

Pi  :  Mp  — »  It. 

The  various  pi  can  be  collected  into  a  single  payoff  function 


p  :  P  x  Mp  — >  JR. 


(3) 


Wc  will  identify  a  game  or  interaction  g  with  the  associated  payoff  function  as  in  (3). 

Wc  will  denote  by  ms  an  clement  of  Ms]  this  is  a  collective  move  for  the  players  in 
5.  To  rns  (i  Ms  and  <  Ms  correspond  an  element  m  of  M r.  The  payoff  function  in 
(3)  can  now  lie  interpreted  by  noting  that  for  (t,rn)  (:  P  x  Mp,  p(i,  rh)  is  the  payoff  to 
player  i  if  the  joint  move  m  is  made.  Wc  can  now  sec  that  a  game  is  non-inter  active  if  and 
only  if  it  satisfies: 

rrii-m  ■>  p[i,m)-p(i,ri ); 

in  other  words,  the  payoff  to  each  player  depends  only  on  that  player’s  action. 

In  cases  where  only  two  agents  arc  interacting,  wc  will  use  a  matrix  notation  to 
represent  the  payoff  function.  Here  is  an  example: 


C 

D 

A 

1 

3 

2 

B 

5 

1 

2 

0 

The  first  player  selects  one  of  the  two  rows  by  choosing  A  or  B;  the  second  one  of 
the  columns  by  choosing  C  or  I).  The  payoffs  correspond  to  the  numbers  in  the  associated 
boxes:  if  move  AC!  is  selected,  for  example,  the  row  player  above  would  receive  a  payoff 
of  3  while  the  column  player  would  receive  1.  The  single  entry  “2”  in  the  AD  position 
corresponds  to  an  identical  payoff  of  2  for  both  players. 


Here  is  the  payoff  function  for  a  non- interactive  game: 


C 

D 

A 

5 

3 

1 

3 

B 

5 

1 

1 

Here  is  one  requiring  coordination: 


C 

D 

A 

7 

4 

B 

5 

6 

Games  such  as  this,  where  one  outcome  (AC  above)  is  preferred  by  all  players  to  any  other, 
arc  referred  to  as  “no-conflict  games”  in  the  game  theory  literature  [13]. 

Finally,  here  is  the  prisoner’s  dilemma.  Each  agent  can  “cooperate”  (C)  or  “defect” 

(D): 


C 

D 

C 

3 

5 

0 

I) 

0 

5 

1 

Independent  of  the  other’s  action,  each  agent  is  better  off  defecting  than  cooperating. 
Unfortunately,  the  payoff  resulting  from  mutual  defection  is  worse  for  both  agents  than 
that  corresponding  to  mutual  cooperation. 

§4.  Rationality 

Game  theory  literature  generally  examines  single  games  and  considers  what  it  means 
for  specific  moves  in  these  games  to  be  rational.  “Rational”  here  is  generally  only  de¬ 
fined  informally,  having  to  do  with  maximizing  the  player’s  return  under  some  ill-defined 
assumptions. 

It  seems  to  me  that,  rather  than  the  particular  move  being  rational  (or  not),  it  is 
the  analysis  leading  to  the  move  which  should  be  examined.  So  I  will  try  to  develop  a 
framework  that  lets  me  discuss  these  analyses  rather  than  simply  the  moves  they  select. 

Let  P  be  a  fixed  group  of  players.  Given  a  game  g,  let  us  denote  by  <7,  the  1  *cvcs  for  t 
which  are  legal  in  g.  Wc  also  denote  the  set  of  all  games  by  G,  and  take  Gj  to  be  UaeG’  9i> 
so  that  Ci  is  the  collection  of  all  moves  for  i  which  arc  legal  in  some  game.  We  will  assume 
throughout  that  any  game  obtained  by  permuting  the  players  in  sonic  particular  game 
g  £  G  is  also  in  G,  so  that  G*  —  Gj  for  all  t  and  j. 


We  now  define  a  decision  procedure  for  player  i  to  be  a  function 


Di-.G -  Gi 

such  that  Di(g)  <E  </i  for  all  g  G  G.  In  other  words,  D,  assigns  to  each  game  a  spcciGc  legal 
move  for  i  in  that  game  and  therefore  encodes  the  process  whereby  i  selects  his  move.  For 
a  fixed  group  S  of  players,  we  denote  nigs  A  by  Ds .  Thus 

ds(o)  =  n 

iCS 

Ds  is  the  “collective”  decision  procedure  used  by  the  players  in  S  to  select  a  joint  move. 

Suppose  now  that  player  i  uses  his  decision  procedure  Z),  to  select  a  move  m,  in  a 
game  g,  and  that  p  is  the  payoff  function  associated  to  this  game.  The  other  players  in 
the  game  use  their  collective  decision  procedure  Dy  to  select  a  move  rrij  and  the  resulting 
payoff  to  i  is  therefore 

pay(i,Diyg)  =  p(i,m).  (4) 

The  function  pay  here  gives  the  payoff  to  t  in  the  game  g  if  he  uses  the  decision  procedure 
Di\  note  that  the  appearance  of  m  in  (4)  means  that  pay  is  implicitly  a  function  of  all  of 
the  £),■’ s,  .and  not  just  Z)j. 

If  we  knew  the  expected  distribution  of  games,  we  could  amalgamate  (4)  over  all 
games  to  get  the  expected  payoff  corresponding  to  the  decision  procedure  Z?t  generally. 
Although  this  is  an  extremely  strong  assumption,  we  will  have  use  of  it,  so  here  goes:  We 
need  a  density  function  p ,  which  assigns  to  each  game  g  0  G  its  likelihood  of  occurring; 
Pti'j)  ~  1-  The  expected  payoff  corresponding  to  D,  is  now  given  by 

pay{i,  Di )  =  £  pi{g)pay(i,  Diyg).  (5) 

ver.’ 


Again,  this  has  implicit  dependence  on  the  decision  procedures  of  the  other  players  as  well. 

Under  these  strong  assumptions,  it  is  clear  that  rationality  corresponds  to  maximizing 
(5).  The  idea  we  want  to  capture  is  that  a  decision  procedure  is  irrational  if  it  results  in 
a  payoff  that  is  provably  subopliinal — recall,  though,  that  the  implicit  dependence  of  (5) 
on  Df  leads  to  a  potential  ambiguity  in  a  definition  such  as  that  Di  is  irrational  if  there 
exists  another  decision  procedure  C,  such  that 

pay{i,  Di)  <  pay(i,Ci). 

We  will  resolve  this  ambiguity  by  quantifying  over  the  CV  and  D,  which  appear  im¬ 
plicitly  in  the  above  equation,  but  in  doing  so,  want  to  allow  for  the  possibility  of  encoding 
some  restrictions  on  the  decision  procedures  of  the  other  players.  Retaining  as  much  gener¬ 
ality  as  possible,  we  will  assume  the  existence  of  sonic  predicate  allowed(Ci,  D{),  and  now 
say  that  a  decision  procedure  D,  is  globally  irrational  (with  respect  to  allowed)  if  there 
exists  a  decision  procedure  C,  such  that,  for  all  C\  and  Dt  with  allowed[Cxy  Di), 

D.)  <  pay(',C,).  (6) 
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In  general,  of  course,  the  density  function  />,  will  not  be  known;  we  therefore  need  a 
notion  of  rationality  depending  only  upon  the  local  payoffs  appearing  in  (4).  To  this  end, 
we  define  a  decision  procedure  D,  to  be  locally  irrational  (or  simply  irrational),  again  with 
respect  to  allowed,  if  there  exists  a  decision  procedure  C,  such  that,  for  all  Cf  and  with 
allowed(Cj,  D i), 

pay(i,  Di,g)  <  pay(i,  C„  g)  (7) 

for  all  games  g ,  with  the  inequality  being  strict  in  at  least  one  case.  In  other  words,  a 
decision  procedure  is  irrational  if  there  is  another  decision  procedure  which  is  better  in 
some  specific  game,  and  no  worse  in  all  others.  We  will  call  a  decision  procedure  rational 
(or  locally  rational )  if  it  is  not  irrational,  and  globally  rational  if  it  is  not  globally  irrational. 

Note  that  we  do  not  call  a  decision  procedure  irrational  merely  because  it  does  not 
achieve  an  optimal  payoff  in  every  game:  it  is  only  if  a  better  payoff  could  be  achieved  in 
some  game  without  reducing  the  payoffs  in  other  games  that  the  definition  applies.  Indeed, 
we  shall  sec  in  section  8  that  there  arc  situations  where  it  is  in  fact  necessary  to  accept  a 
suboptimal  payoff  in  certain  games. 

Lemma  4.1.  Any  globally  rational  decision  procedure  is  locally  rational. 

Proof.  For  any  allowed  decision  procedures  C,  and  D*,  summing  (7)  over  all  games  re¬ 
produces  (6).  □ 

The  results  of  the  next  section  deal  with  conditions  which  must  be  satisfied  by  any  locally 
rational  decision  procedure.  The  point  of  the  lemma  is  that  a  globally  rational  decision 
procedure  must  meet  these  requirements  as  well. 

The  nature  of  the  definitions  of  rationality  depends,  of  course,  on  the  allowed  pred¬ 
icate.  If,  for  example,  we  knew  the  decision  procedures  of  the  other  players,  we  could 
restrict  allowed  to  these  decision  procedures,  so  that  our  decision  procedure  could  be  eas¬ 
ily  determined  by  the  definition  of  local  rationality  (7).  Essentially,  we  would  be  moving 
with  complete  knowledge  of  the  other  players’  choices  of  action. 

In  practice,  of  course,  this  will  not  be  the  case,  and  allowed  will  hold  for  a  variety 
of  decision  procedures  other  than  the  ones  the  other  players  arc  actually  using.  This  will 
lead  to  some  freedom  in  our  own  (presumed  rational)  choice  of  decision  procedure;  our 
interest  will  be  in  determining  the  nature  and  amount  of  this  freedom  given  a  variety  of 
(incomplete)  assumptions  regarding  the  decision  procedures  of  the  other  players  or  agents. 

At  this  point,  we  have  made  no  such  assumptions  at  all.  The  fact  that  allowed  is  a 
function  of  two  decision  procedures,  for  example,  allows  us  to  consider  situations  where 
the  decision  procedures  of  the  other  agents  are  not  independent  of  our  own.  In  fact,  the 
•ability  to  drop  this  independence  assumption  is  the  pragmatic  reason  we  arc  working  not 
with  the  moves  themselves,  but  with  the  analyses  which  lead  to  them.  13y  working  in  this 
fashion,  it  is  possible  to  assume  that  the  agents  have  knowledge  of  each  others’  strategics 
without  being  led  to  the  circular  arguments  that  otherwise  pervade  this  sort  of  analysis. 

As  an  example,  consider  the  following  well-known  “paradox”:  an  alien  approaches  you 
with  two  envelopes,  one  marked  “$”  and  the  other  ‘Y”.  The  first  envelope  contains  some 
number  of  dollars,  and  the  other  the  same  number  of  cents.  The  alien  is  prepared  to  give 
you  the  contents  of  either  envelope. 
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The  catch  is  that  tlie  alien,  who  is  omniscient,  is  aware  of  the  choice  you  will  make. 
In  an  attempt  to  discourage  greed  on  your  part,  he  has  decided  to  put  one  unit  of  currency 
in  the  envelopes  if  you  will  pick  “$”,  but  one  thousand  units  if  you  will  pick  af.  Bearing 
in  mind  that  the  alien  has  decided  upon  the  contents  of  the  envelopes  before  you  pick  one, 
which  should  you  select?  (If  this  example  is  insufficiently  compelling  for  an  AI  audience, 
multiply  all  of  the  figures  by  a  million.) 

Here  arc  the  payoffs  for  this  game;  the  alien’s  payoffs  simply  reflect  his  desire  to  teach 
you  the  desired  lesson.  (He  can  make  all  the  money  lie  needs  in  the  stock  market,  anyway.) 


1000 

1 

$ 

0 

1000. 

1 

1. 

i 

1 

10. 

0 

0.01 

Since  the  payolf  for  $  is  greater  than  that  for  for  either  of  the  alien’s  options,  any  of  the 
conventional  game-theoretic  analyses  will  lead  us  to  select  the  dollar  envelope,  and  we  will 
presumably  receive  only  $1  as  a  result. 

The  decision  procedure  paradigm  is  flexible  enough  to  handle  this  situation.  If  we  are 
player  1  and  the  alien  is  player  2,  we  have  that  the  only  allowed  D%  is: 

r>  (n\  -  I  1000>  if  D^d)  =  ^  *nd  fat 

if  Dt(g)=$.  (8) 

Inserting  this  into  (4)  gives  pay(l,f,g)  =  10  and  pay(l,$,g)  =-  1,  allowing  us  to  conclude 
from  (7)  that  it  is  indeed  irrational  to  pick  $  in  this  game. 

In  no  way  have  we  avoided  the  paradox  of  the  alien’s  omniscience;  we  have  merely 
found  a  way  to  describe  this  omniscience  in  the  decision  procedure  (8).  We  wit!  see  in  the 
next  section  that  the  “case  analysis”  argument  which  would  have  led  us  to  select  $  in  the 
above  game  is  valid  if  the  decision  procedures  of  the  various  players  arc  independent. 

It  is  a  small  step  from  the  scenario  we  have  described  to  one  with  a  more  serious 
difficulty.  Suppose  that  the  player  1  in  the  above  game,  instead  of  being  you  or  I,  is 
another  omniscient  alien.  The  new  alien,  in  an  attempt  to  encourage  the  original  one  to 
be  more  cautious  with  its  money,  decides  to  select  $  if  the  offering  alien  puts  one  thousand 
units  of  currency  in  the  envelopes,  and  to  select  otherwise.  What  happens? 

Here  arc  the  payoffs  for  the  new  game: 


1000 

1 

$ 

0 

1 

1 

0 

i 

1 

0 

0 

1 

The  situation  appears  to  be  circular;  in  fact  the  descriptions  of  the  two  aliens  are 
inconsistent.  The  decision  procedure  of  the  new  alien  is  supposedly  given  by: 


lh 


i, 

$, 


if  V-tio)  -  1,  and 
if  Ih{<j)  ---  1000. 
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This  in  in  clear  conflict  with  (8),  in  that  there  are  no  allowed  C,  or  D f  for  this  game.  (Is 
this  example  an  .argument  for  monotheism?) 

We  will  be  interested  in  a  variety  of  assumptions  regarding  the  decision  procedures 
of  the  other  agents;  they  will  fall  into  two  distinct  types.  The  first  consists  of  rationality 
assumptions:  we  can  assume  that  the  other  agents  are  locally  or  globally  rational.  Alter¬ 
natively,  we  can  make  behavior  assumptions,  assuming  that  the  other  agents’  behavior  is 
independent  of  our  own,  or  that  it  is  the  same  as  our  own. 

Let  us  deal  with  the  rationality  assumptions  first.  We  will  say  that  a  joint  decision 
procedure  Dg  is  rational  if  it  is  the  product  of  individual  decision  procedures  which  are 
all  rational,  and  now  can  define: 

la.  Separate  (local)  rationality:  CV  not  locally  rational  =>  -> allowed{Ci ,  Dt),  and  sim¬ 
ilarly  for  Dj. 

lb.  Separate  global  rationality:  C*  not  globally  rational  =>■  ->a//owed(CV,  Dj),  and 
similarly  for 

The  appearance  of  allowed  in  the  rationality  definitions  may  appear  to  make  these  last 
definitions  circular,  but  this  is  not  the  case.  The  reason  for  this  is  that  as  the  allowed  set 
gets  smaller,  more  and  more  decision  procedures  will  be  defined  to  be  irrational  by  (6)  and 
(7).  The  separate  rationality  assumptions  can  therefore  be  used  to  produce  increasingly 
sharp  limits  on  the  decision  procedures  of  the  agents  involved.  This  can  be  formalized  by 
defining  the  set  of  rational  decision  procedures  for  player  i  to  be  the  maximal  fixed  point 
of  an  operator  defined  in  terms  of  (6)  or  (7).  (The  example  following  corollary  5.2  is  an 
example  of  this.) 

The  behavior  assumptions  are  more  difficult  to  capture  formally.  When  we  say,  “the 
other  agents  will  behave  as  I  do,”  for  example,  we  mean  that  the  other  agents  would  act 
simiiary  were  they  to  find  themselves  in  my  current  situation. 

This  notion  depends  on  that  of  “permuting”  the  players  in  a  game  in  order  lo  examine 
that  game  from  another  player’s  point  of  view.  In  order  to  capture  this  idea,  suppose  that 
there  arc  n  players  in  P,  and  denote  by  S„  the  group  of  permutations  on  an  n-clcmcnt 
set.  Given  a  game  with  payofr  function  p,  and  a  fixed  permutation  a  £  S„ ,  we  can  define 
a  permuted  game  p„  with  payofr  function  given  by 

=  P(b™)- 

Since  we  arc  identifying  games  and  payofT  functions,  it  follows  from  this  that  a  permutation 
o  induces  a  mapping  (also  to  be  denoted  cr)  from  games  to  games. 

To  get  a  feel  for  this,  we  might  say  that  a  decision  procedure  C,  is  unbiased  if  C{[g)  = 
Ci(a(;/))  whenever  o(i)  i.  In  other  words,  a  shuffling  or  the  other  players  in  the  game 
does  not  affect  i’s  choice  of  action;  he  is  uniform  in  his  treatment  of  the  other  agents. 

Slightly  more  generally,  we  might  fix  S  C  P  and  assume  that  aC.s(ff)  --  Cs (<?({/)) 
whenever  <r(S)  -  6’.  The  implication  of  this  is  not  only  that  the  group  S’  is  unbiased 
toward  the  remaining  players  in  S,  but  also  that  the  behaviors  of  the  individuals  within 
the  group  are  identical  in  the  above  sense. 

We  now  define: 

2a.  Common  behavior:  If,  for  some  <j,  aCr{tl)  /  Cr{v{fj)),  then  'allowcd(Ci}  /},),  and 
similarly  for  D,-. 


This  assumption  (which  cannot  be  described  in  the  conventional  game-theoretic  for¬ 
malism)  is  peculiarly  appropriate  to  an  AI  setting;  it  would  be  straightforward  to  equip 
potentially  interacting  agents  with  matching  decision  procedures.  We  will  see  in  section  6 
that  it  is  in  fact  sufficient  to  equip  the  agents  with  only  the  common  behavior  assumption 
(theorem  6.2),  and  also  that  this  is,  at  least  in  some  sense,  a  good  idea  (theorem  6.3). 

We  also  have: 

2b.  Independent  behavior:  Cx  Dx  =>  -<allowed(Ci,  Dx).  This  amounts  to  assuming 
that  each  agent’s  decision  procedure  does  not  influence  the  others.  This  assumption 
is  generally  made  in  the  game  theory  literature,  and  is  equivalent  to  assuming  that 
the  moves  of  the  other  players  arc  fixed  in  advance  (since  their  decision  procedures 
are),  although  the  exact  nature  of  these  moves  remains  unknown. 

Separate  rationality  and  independent  behavior  arc  independent  but  consistent;  there 
are  decision  procedures  which  are  rational  under  one  assumption  but  irrational  under  the 
other.  We  will  refer  to  the  combination  of  separate  rationality  and  independent  behavior 
as  individual  rationality  (or  perhaps  as  individual  global  rationality) . 

Lemma  4.2.  Common  behavior  implies  separate  rationality  if  at  least  one  of  the  players 
has  a  rational  decision  procedure. 

Proof.  If  some  decision  procedure  Dx  leads  to  a  payoff  pay(i,D,,g)  =  p(i,m)  for  player  i 
under  the  common  behavior  assumption,  the  payoff  to  some  other  player  j  —  o(i)  in  the 
permuted  game  o(g)  will  be 

Pa y{h  Dj,a{g))  =  p„{a{i)xDj{a{g))) 

=  Prr{a{i),(r{D,{g))) 

~  V{i,Oi{<j))  =  pay{hDt,g). 

It  follows  that  if  a  decision  procedure  C,  produces  a  universal  improvement  Tor  i,  the  per¬ 
mutation  oC,o~i  will  produce  a  universal  improvement  for  j.  Thus,  if  Dt  were  irrational 
for  z,  oDto~i  would  be  irrational  for  j.  The  conclusion  follows.  □ 

Note  tha1  if  seems  impossible  to  strengthen  lemma  4.2.  For  example,  it  is  not.  nec¬ 
essarily  the  case  that  a  decision  procedure  that  is  rational  for  one  agent  is  rational  for  all 
others.  The  presence  of  a  third  player  whose  actions  are  known  to  be  biased  in  favor  of 
one  of  the  others  can  introduce  asymmetry  of  this  sort — we  approach  problems  differently 
if  there  are  additional  agents  present  upon  whose  support  and  cooperation  we  can  rely. 

It  is  also  not  the  case  that  common  behavior  implies  separate  global  rationality.  Dif¬ 
fering  density  functions  can  easily  result  in  a  decision  procedure  that  leads  to  a  global 
improvement  for  one  agent  while  being  a  disaster  for  another.  Suppose,  however,  that  we 
say  that  the  various  density  funefio  is  />,  match  if  p, ,(,)<?  -  px  for  all  a  C_  Su.  We  then  have 
the  following: 

Lemma  4.3.  Common  behavior  implies  separate  global  rationality  if  at  least  one  of  the 
]>laycrs  has  a  globally  rational  decision  pi  edurc  and  all  of  the  players  have  matching 
density  functions. 

Proof.  If  the  density  functions  match,  the  global  \yoffs  to  all  of  the  players  arc  identical; 
the  conclusion  follows.  (The  proof  that  the  global  p.  offs  are  identical  is  embedded  in  the 
proof  of  theorem  6.3.)  □ 


We  will  refer  to  the  combination  of  common  behavior  and  separate  rationality  as 
common  rationality  (or  perhaps  as  common  global  rationality). 

A  few  more  words  about  matching  density  functions  are  probably  in  order  before  we 
return  to  local  issues.  If  we  are  interested  in  situations  where  all  of  the  agents  agree  about 
the  likelihood  of  various  interactions,  then  the  density  functions  will  match  if  and  only 
if  p  —  po  for  all  o,  so  that  the  likelihood  of  a  game  is  not  changed  by  permuting  it.  In 
other  words,  equal  density  functions  will  match  just  in  case  each  agent  is  as  likely  to  find 
himself  in  any  particular  situation  as  is  any  other.  The  more  general  formulation  amounts 
the  statement  that  the  agents  believe  themselves  to  be  equally  likely  to  find  themselves 
in  specific  interactions — for  example,  each  might  believe  that  interactions  unfavorable  to 
him  were  more  likely  than  others.  These  differing  density  functions  would  still  match  if 
the  amount  of  pessimism  were  uniform  among  the  various  agents. 

§5.  local  rationality 

In  this  section  we  investigate  the  consequences  of  our  definitions  of  rationality.  We 
will  assume  throughout  that  g  is  some  fixed  game  with  payofT  function  p. 

We  define  a  single  move  m,  to  be  rational  in  a  game  g  if  there  is  a  rational  decision 
procedure  Di  with  D,(g)  =  mt  ,  and  now  have: 

Theorem  5.1  (Case  analysis).  Assuming  independent  behavior,  if  for  some  moves  c* 
a lid  for  all  mit 

p(i,d{  x  mr)  <  p(i,Ci  x  mi), 

then  di  is  not  rational  in  g. 

Proof.  Suppose  that  Di(g)  -  dt,  and  let  Ct  be  the  decision  procedure  given  by 

cm  =  { IW)’  ***« 

l  C»,  I'  9  —  9- 

For  any  allowed  C,  and  D,,  C,  —  £),;  it  follows  that  if  we  set  m,  =  C,{g)  —  D,(g), 

pay{i,Di,g)  =  p{i,d{  x  mf)  <  p(t',c,  x  mT)  =  pay(i,Citg), 

while  pay(i,Di,g')  =  pay(i,Ci,g')  for  g'  ^  g,  so  that  the  decision  procedure  Di  is  irra¬ 
tional.  a 

It  is  an  invalid  application  of  this  theorem  that  led  us  to  choose  $  in  the  paradox  of 
the  last  section.  Of  course,  the  formulation  of  the  interaction  there  is  such  as  to  indicate 
clearly  that  the  independent  behavior  assumption  does  not  hold. 

The  dependence  of  the  proof  on  the  independent  behavior  assumption  lies  in  part  in 
the  statement  that  pny(i,Di,g')  -  poyfi.O,,#')  for  g'  /-  g.  In  order  to  conclude  this,  we 
needed  to  know  that  the  other  agents  would  not  change  their  actions  in  g'  in  response  to 
our  selecting  the  move  c,  in  g. 


Corollary  5.2  (Iterated  case  analysis).  Assuming  individual  rationality,  if  for  some 
moves  Cj  and  d,,  for  all  rational  m,, 


p(i,di  x  mt-)  <  p(i,Ci  x  mr), 

then  d,  is  not  rational  in  g. 

Proof.  Clear.  a 

If  this  result  allows  us  to  conclude  that  there  is  a  unique  rational  move  in  some  game 
g,  this  is  known  as  the  solution  in  the  complete  weak  sense  in  the  game  theory  literature. 

The  consequences  of  case  analysis  are  quite  well  known.  Consider  the  example  used 
to  introduce  our  payoff  matrix  notation: 


C 

D 

A 

1 

3 

2 

B 

5 

1 

2 

0 

Independent  of  the  column  player’s  choice,  the  row  player  will  be  better  ofT  if  he  makes 
move  A,  since  his  payoff  will  be  3  as  opposed  to  2  if  the  other  player  selects  C,  and  2  as 
opposed  to  0  if  D  is  chosen.  Case  analysis  therefore  implies  that  A  is  the  only  rational 
move  for  the  row  player  in  this  game. 

There  is  no  such  implication  for  the  column  player.  Assuming  individual  rationality, 
however,  he  will  realize  that  the  row  player  can  be  counted  on  to  make  move  A,  and  will 
therefore  respond  with  I)  (receiving  a  payoff  of  2  instead  of  1). 

Note  that  the  column  player  would  have  bcnclitted,  at  no  cost  to  the  row  player,  if 
I1C  had  been  selected  instead  of  AI).  The  prisoner’s  dilemma  is  an  extension  of  this  idea: 


C 


D 


Case  analysis  leads  each  player  to  defect,  leading  to  a  result  which  is  unfortunate  for  both 
of  them. 

The  difficulty  is  a  consequence  of  the  fact  that  the  agents  arc  pursuing  their  goals  too 
blindly:  they  arc  employing  the  independent  behavior  assumption  in  a  situation  where  it  is 
inadvisable  to  do  so.  In  general,  the  consequences  of  theorem  5.1  will  make  the  independent 
behavior  assumption  inconsistent  with  any  cooperative  or  altruistic  behavior. 

Common  behavior  provides  an  answer: 

Theorem  5.3  (Cooperation).  Assume  common  behavior  and  local  rationality,  and  sup- 

-♦ 

pose  that  g  lias  moves  c  and  d  such  that 


for  all  players  j,  with  the  inequality  being  strict  in  at  least  one  case.  Then  if  P  denotes  the 
set  of  all  agents  in  the  interaction,  and  c  is  possible  as  the  outcome  of  a  common  decision 
procedurefor  the  agents  in  P,  Dp(g)  ^  d .  In  other  words,  the  collective  move  d  is  avoided. 

Proof.  Suppose  that  Dp(g)  =  d,  so  that  77,(g)  =  d,  for  all  players  i.  Now  let  G'  be 
the  set  of  all  games  obtained  by  permuting  the  players  in  P,  and  consider  the  decision 
procedure  (7,  obtained  by  modifying  Z?,  so  that  the  move  c  and  its  permuted  versions  are 
obtained  for  the  games  in  G' .  The  hypotheses  of  the  theorem  will  allow  us  to  apply  (7)  to 
conclude  that  D ,  is  irrational.  □ 

In  other  words,  if  mutual  defection  is  to  everyone’s  disadvantage,  then  at  least  one 
player  will  cooperate.  For  an  interaction  such  as  the  prisoner’s  dilemma  which  is  symmetric 
(in  that  pa  =  p  for  all  <r),  it  follows  that  all  of  the  players  cooperate.  This  conclusion  has 
an  analog  in  the  informal  arguments  of  [4]  and  [9]. 

No-conflict  games  are  handled  as  a  special  case  of  this  result: 

Corollary  5.4  (Coordination).  Assume  common  behavior,  and  suppose  that  g  has  a 
move  c  such  that  for  any  d  ^  c, 


PU,  c)  >  p[j,  d) 

for  all  players  j,  with  the  inequality  being  strict  for  at  least  one  j.  Then  Dp[g)  =  c  . 

Proof.  Apply  theorem  5.3  to  eliminate  each  of  the  alternatives.  □ 

Returning  to  5.3  itself,  note  that  the  theorem  does  not  imply  that  all  of  the  players 
cooperate;  the  possibility  of  one  agent  cooperating  while  the  others  defect  (undoubtedly 
beneficial  to  the  defectors  and  disastrous  for  the  cooperator)  is  typical  of  the  “.altruism” 
allowed  under  the  common  behavior  assumption.  Consider  the  following  non-result: 

Non-theorem  5.5  (Restricted  case  analysis).  Assume  minimal  rationality,  and  sup¬ 
pose  that  there  exist  c,  and  d,  such  that,  for  all  cf  and  df, 

p(t,  d)  <  p(i,  c). 


Then  d ,  is  irrational  in  g. 

Proof?  If  Di(g)  —  di,  consider  the  decision  procedure  given  by 

CM)  =  ( DiW)'  *?' * 

( c,,  otherwise. 

This  leads  to  pay(i,Ci,g)  >  Pay{h  D,,  g)  independent  of  the  nature  of  the  allowed  predi¬ 
cate.  □ 

The  problem  is  that  we  cannot  show  that 


pay(*,<W)  >  pay{i,Di,g') 
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(9) 


for  g'  7^  g.  Independent  behavior  allows  us  to  conchide  this,  of  course;  witness  theorem 
5.1.  But  under  the  common  behavior  assumption,  if  the  “better”  move  c,  forces  a  reduced 
payoff  for  some  other  player,  then  (9)  will  specifically  not  hold  for  some  permutation  of 
g.  This  of  course  does  not  mean  that  C{  is  irrational;  it  merely  means  that  C*  cannot  be 
used  to  prove  that  D{  is. 

Restricted  case  analysis  is  in  fact  independent  of  the  assumption  of  common  behavior, 
although  it  is  consistent  with  it. 

§6.  Global  rationality 

In  this  section  we  will  investigate  the  interplay  between  the  common  behavior  assump¬ 
tion  and  the  definition  of  global  rationality  (6).  Before  doing  so,  however,  consider  the 
following  symmetric  game: 


A 

B 

A 

0 

1 

B 

1 

0 

Common  behavior  docs  very  poorly  here, 

resulting  in  a  payoff  of  0  for  both  players. 

The  difficulty  lies  in  the  symmetry — if  the  payoffs  are  replaced  by  1  +  c  and  1  —  e,  this 
difficulty  will  be  avoided.  We  therefore  define  a  game  g  with  payoff  function  p  to  be  locally 
disambiguated  (or  simply  disambiguated)  if,  p„  /-  p  for  every  non-trivial  permutation  a  of 
the  players  in  g. 

Recall  that  we  apply  the  permutation  a  to  both  the  player  and  the  move.  Thus  the 
following  (symmetric)  game  is  not  locally  disambiguated: 


A 

B 

A 

0 

2 

1 

B 

1 

2 

0 

while  this  game  is  locally  disambiguated: 


A 

B 

C 

0 

2 

1 

D 

2 

1 

0 

To  sec  that  this  last  game  is  not  symmetric,  all  wc  need  do  is  note  that  the  column  player 
has  a  fundamental  advantage  in  it. 

Note  also  that,  in  the  presence  of  the  common  behavior  assumption  Dj  —  <r£)l<r~1, 
the  payoir  function  (4)  has  no  unexpressed  dependence  on  the  decision  procedures  of  the 
other  players,  since  there  is  already  an  explicit  dependence  on  Dt  in  pay{i,  I)itg). 


For  a  fixed  game  g,  we  define  the  orbit  of  g,  denoted  o(y),  to  be  the  set  of  all  games 
obtained  by  permuting  the  players  in  g.  We  also  extend  the  payoff  function  to  orbits, 
defining 

pay[hDi,g)=  pi{g)pay(i,Dug').  (10) 

A  game  g  will  be  called  globally  disambiguated  if  pay(i,  Dt,  g)  ^  pdy{i,Ci,g)  whenever 
Di(g)  Ci{g).  Note  that,  as  a  result  of  the  appearance  of  p,  in  (10),  this  definition  is 
dependent  upon  the  values  of  the  density  function. 

Theorem  6.1.  Assume  common  behavior.  Then  there  is  a  unique  globally  rational  move 
in  any  globally  disajnbiguated  game. 

Proof.  We  can  assume  without  loss  of  generality  that  G,  the  set  of  all  games,  is  equal  to 
o(p)  for  some  single  game  g.  Now  pay(i,Di)  =  pdy(i,Di,g).  Let  {CJ  be  the  collection 
of  all  decision  procedures  which  maximize  this  payoff;  the  global  disambiguation  ensures 
that  the  6\’s  agree  on  g.  Ci(g)  is  therefore  the  unique  globally  rational  move  in  g.  o 

This  is  in  some  sense  a  surprising  result,  since  it  does  not  require  the  players  to  have 
common  knowledge  of  the  payoff  function  (in  that  each  agent  knows  that  the  other  agents 
know  that  it  knows  the  payoir  function,  etc.),  llalpern  and  Moses  refer  to  this  situation, 
where  everyone  knows  p,  but  without  the  further  requirements  of  full  common  knowledge, 
as  E-knowledge.  In  light  of  their  result  [8]  that  it  is  virtually  impossible  for  a  distributed 
system  to  achieve  common  knowledge,  this  is  an  important  consideration. 

The  following  is  an  immediate  consequence  of  theorem  6.1: 

Theorem  6.2.  Suppose  that  each  agent  in  a  globally  disambiguated  interaction  makes 
the  assumption  or  common  behavior,  and  that  the  density  Functions  of  the  various  agents 
match.  Then  separate  global  rationality  implies  that  the  common  behavior  assumption 
will  in  fact  be  valid. 

Proof.  Theorem  6.1  says  that  each  agent  will  have  at  most  one  globally  rational  deci¬ 
sion  procedure  under  these  assumptions;  lemma  1.3  guarantees  that  this  globally  rational 
decision  procedure  is  the  one  which  satisfies  the  common  behavior  assumption.  o 

In  other  words,  we  do  not  need  to  equip  interacting  agents  with  decision  procedures 
which  are  in  fact  permutations  of  one  another;  if  we  provide  them  merely  with  the  assump¬ 
tion  of  common  behavior  and  with  instructions  to  behave  in  a  globally  rational  fashion, 
their  decision  procedures  will  match  as  a  result. 

Recall  that  a  move  m  in  a  game  is  Pareto  optimal  if,  for  any  move  n,  there  is  a 
player  %  for  whom  />(*,  ri)  <  p(i,tn).  The  content  of  theorem  5.3,  for  example,  is  that 
Parcto-suboplinial  moves  arc  irrational  under  the  assumption  of  common  behavior. 

Consider  now  a  meta-game  M  in  which  a  “move”  for  player  i  is  the  selection  of  a 
decision  procedure  l)t,  so  that  a  joint  move  is  the  selection  of  a  joint  decision  procedure 
D<j.  We  define  the  payoffs  in  M  by 

p[i,Dr)  ~  pay{i,Di)\ 

this  is  the  payofT  to  i  of  selecting  decision  procedure  /},  (if  the  other  players  select  the 
decision  procedure  D,).  The  main  result  of  this  section  is  the  following: 
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Theorem  6.3.  If  all  players  have  matching  density  functions,  and  all  games  in  G  are 
locally  disambiguated,  then  any  collective  decision  procedure  I)p  which  obeys  the  common 
behavior  assumption  and  is  globally  rational  for  the  players  in  P  will  be  Pareto  optimal 
in  the  meta-game  M. 

Proof.  Again,  we  can  assume  without  loss  of  generality  that  G  —  o(g)  for  some  fixed  game 
g  with  payolT  function  p. 

Under  the  assumption  of  common  behavior,  there  will  be  a  single  move  c  which  is 
made  in  g,  with  permutations  of  c  being  made  in  permutations  of  g.  Now  fix  a  player  t; 
we  will  denote  by  C,  the  decision  procedure  which  generates  the  move  c. 

The  payoff  to  i  of  the  decision  procedure  C,  is  now  given  by 

pay(i,  Ci)  =  Pi  (s' )  P«S( * .  g ') 

g'ec, 

~  Y2  Pi(<T)Pay(i>Ci><,{9)) 

<7 

=  (•»*(*)) 
a 

=  5Zp‘(,7)p(a_1(i)»c)- 

a 

The  local  disambiguation  assumption  guarantees  that  every  move  c  is  available  as  the 
outcome  of  a  commonly  rational  decision  procedure;  (7,  will  therefore  be  globally  rational 
just  iu  case 

Y2pAa)p(°'{  (*)>«)  ^  (n) 

a  a 

for  all  moves  d  in  g. 

If  C{  were  not  Pareto  optimal,  there  would  be  a  collection  of  moves,  one  for  each  game 
in  G  (i.e.,  for  each  a  e  Sn),  leading  to  better  global  payoffs  for  all  of  the  players  in  g.  If 
we  denote  these  moves  by  ma ,  the  payoff  to  t  will  be 

ft  (<0p*  (*»"**)• 

oeSn 

Meanwhile,  the  payoffs  to  another  player  rr'(t)  will  be 

Y2 


It  follows  that  C(  will  not  be  Pareto  optimal  only  if 


>  YLpi(a"Wa"- ^(*)>*) 


m 


-  ’ ' 


for  all  o' ,  with  the  inequality  being  strict  at  least  once.  (The  global  payoff  under  common 
behavior  does  not  change  from  player  to  player).  This  implies 

?)■  (i2) 

a'  o  an 

The  left  hand  side  of  (12)  can  be  easily  rewritten  as 

a  o'  at j' 

If  we  write  a"  for  <r'~la  and  m'a  for  o-1(mCT),  this  is  now 

a  a1  ff  ou 

a  a" 

=  n!5^/><(a")p(<7,,_1(t),c) 
a" 

This  is  in  conflict  with  (12),  and  the  proof  is  complete.  a 

I  cannot  resist  the  temptation  to  rephrase  this:  if  all  men  are  indeed  created  equal, 
the  Golden  Rule  is  valid.  Well,  at  least  it’s  Pareto  optimal. 

§7.  Communication 

We  seem  to  have  drifted  rather  far  from  the  concerns  of  DAI.  This  is  not  the  case, 
though;  the  point  of  this  paper  is  the  following: 

Proposal.  Intelligent  agents  should  be  equipped  with  the  common  behavior  assumption. 

There  arc  a  variety  of  reasons  for  this.  The  first  is  the  Pareto  optimality  of  common 
behavior,  us  in  theorem  6.3.  Provided  that  the  agents  being  considered  expect  to  encounter 
a  similar  distribution  of  interactive  situations,  there  is  no  assumption  or  set  of  assumptions 
which  would  make  the  agents  uniformly  more  effective  problem  solvers. 

It  can  be  argued  that  the  assumption  of  matching  density  functions  is  an  invalid  one, 
and  this  is  a  fair  criticism.  In  the  presence  of  specific  information  regarding  differences  in 
the  density  functions  of  various  agents,  it  is  possible  that  more  effective  assumptions  can 
be  found.  But  in  many  cases,  the  strength  of  the  local  results  in  section  5  may  be  sufficient 
to  ensure  a  satisfactory  result  if  the  common  rationality  assumption  is  adopted. 

Additional  advantages  arc  a  consequence  or  theorem  6.1.  Provided  that  interacting 
agents  can  agree  on  a  globally  disambiguated  payoff  function  for  a  game  g,  they  need  com¬ 
municate  no  further.  This  is  a  consequence  of  the  fact  that  this  payofr  function,  in  combi¬ 
nation  with  the  common  behavior  assumption,  will  be  sufficient  to  determine  the  agents’ 
actions. 


There  are  a  variety  of  advantages  to  restricting  communication  in  this  fashion.  Firstly, 
it  is  possible  to  establish  uniform  protocols  for  the  exchange  of  information  regarding 
payoff  matrices.  One  such  protocol  would  simply  have  the  agents  broadcast  their  version 
of  the  payoff  function  in  sequence;  as  soon  as  there  is  uniform  agreement  on  a  value,  the 
interaction  could  proceed. 

The  reason  that  we  arc  able  to  get  by  without  common  knowledge  of  the  payoff 
function  will  be  a  bit  clearer  if  we  return  to  our  widget  example.  Let  us  suppose  that  the 
Palo  Alto  robot  broadcasts  a  payoff  matrix  indicating  the  advisability  of  slopping  to  pick 
up  the  various  tools,  and  that  the  Boston  robot  confirms  this.  At  this  point,  the  robots 
know  the  following: 

(1)  Both  robots  know  that  it  is  to  their  advantage  to  stop  and  acquire  the  tools,  and 

(2)  The  Palo  Alto  robot  knows  that  the  Boston  robot  knows  that  this  is  the  case. 

It  is  not  the  case,  though,  that  the  Boston  robot  knows  that  the  Palo  Alto  robot  is 
aware  of  the  advantages  to  be  gained  by  acquiring  the  tools,  since  the  Palo  Alto  robot  has 
not  yet  acknowledged  the  Boston  robot’s  message  confirming  the  payoff  function  (nor  will 
he);  all  that  the  Boston  robot  knows  is  that  when  the  Palo  Alto  robot  receives  the  Boston 
robot's  last  message,  he  will  be  aware  of  the  values  in  the  confirmed  payoff  matrix. 

The  effect  of  this  is  that  the  Boston  robot  docs  not  know  precisely  when  the  Palo  Alto 
robot  will  be  prepared  to  start  on  his  journey  to  Pittsburgh;  all  he  knows  is  that  when  he 
arrives,  he  will  have  a  borogove  with  him. 

Let  us  return  to  more  general  considerations.  In  the  presence  of  disagreement,  for 
example,  it  will  often  be  possible  for  one  agent  to  prove  its  description  of  the  payoff 
function  to  be  correct.  Such  a  proof  might  simply  explain  the  situation  in  which  that 
particular  agent  finds  itself,  or  might  instead  provide  a  useful  medium  for  agents  to  advise 
one  another  about  possibly  unforeseen  consequences  of  their  activities.  If  the  Boston  robot 
in  our  widget  example  had  been  unaware  of  the  value  of  having  a  zazxyfrax  and  a  borogove 
to  help  with  the  construction,  the  demonstration  by  the  Palo  Alto  robot  that  there  was 
mutual  benefit  to  be  had  by  stopping  and  picking  up  these  tools  would  constitute  advice 
of  just  this  sort. 

The  fact  that  it  is  possible  to  demonstrate  the  accuracy  of  a  payoff  function  will  also 
alleviate  problems  arising  from  the  fact  that  it  may  be  to  a  particular  agent’s  advantage  to 
lie,  as  has  been  remarked  by  Roscnschcin  [15].  It  is  awkward  to  prevaricate  in  a  situation 
where  one  can  be  required  to  vindicate  one’s  claims!  The  only  other  solution  which  has  been 
proposed  to  this,  again  by  Roscnschcin  [14],  is  to  have  agents  exchange  binding  promises, 
but  it  is  not  at  all  clear  how  the  binding  nature  of  these  promises  could  be  effected. 

There  is  one  point  we  have  left  unaddressed,  and  that  involves  the  practical  import 
of  the  disambiguation  assumptions  in  the  theorems  of  the  last  section.  Fortunately,  the 
following  result  should  be  fairly  clear: 

Theorem  7.1.  Every  game  is  arbitrarily  close  to  one  which  is  locally  and  globally  disam¬ 
biguated.  a 

Note,  however,  that  since  in  some  cases  the  decision  procedure  produced  by  theorem 
6.1  may  vary  disconlinuoiisly  with  respect  to  the  payoff  function  for  the  game  in  question, 
it  is  possible  for  a  group  of  intelligent  agents  to  have  difficulty  agreeing  on  a  perturbation 
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that  would  disambiguate  it.  It  docs  not  appear  that  such  difficulties  can  be  addressed 
within  the  framework  we  have  developed. 

§8.  An  application 

Let  me  end  with  an  example  that  will  hopefully  make  the  practical  import  of  these 
ideas  a  bit  clearer.  I  will  imagine  a  very  simple  environment  which  contains  two  processors 
and  one  peripheral  which  can  be  used  by  the  processors  to  speed  their  computation.  One 
of  the  processors  is  initially  responsible  for  dividing  a  problem  into  two  smaller  ones;  it 
then  hands  one  of  these  subgoals  off  to  the  other  machine.  The  peripheral  might  be,  for 
example,  a  floating  point  accelerator  (FPA). 

The  two  processors  (which  I  will  call  simply  1  and  2)  must  now  decide  whether  or  not 
to  request  access  to  the  FPA;  each  processor  has  the  option  of  making  a  request  (R)  or  of 
making  no  request  (N).  In  the  interest  of  simplicity,  I  will  assume  that  there  is  no  facility 
for  arbiting  between  conflicting  requests:  if  both  processors  request  the  peripheral,  both 
arc  denied  access. 

The  payoffs  to  the  two  processors  are  related  inversely  to  the  amount  of  time  that 
they  will  take  to  complete  their  respective  tasks.  I  will  assume,  though,  that  the  payoff  to 
each  processor  is  simply  the  amount  of  time  taken  for  its  computation,  with  the  goal  of 
the  processors  now  being  to  minimize  their  payoffs. 

Initially,  then,  each  processor  receives  a  task,  and  will  determine  what  the  advantage  is 
of  requesting  access  to  the  FPA.  Let  me  assume  that  each  processor  makes  this  calculation 
for  itself  and  transmits  the  results.  If  processor  1  (the  “row”  player)  had  payoffs  of  6 
( without  the  FPA )  .and  2  (with  it),  while  processor  2  had  payoffs  of  4  and  I,  the  payoff 
matrix  would  look  like  this: 


(13) 

Note  first  that  this  game  is  a  version  of  the  prisoner’s  dilemma.  Since  each  processor 
is  no  worse  off  requesting  access  to  the  FPA  than  not,  an  application  of  case  analysis  will 
lead  to  the  unfortunate  choice  {RR}. 

Let  us  suppose  for  the  moment  that  the  density  functions  px  arc  uniform  over  all 
games.  An  examination  of  (11)  now  reveals  that  any  commonly  rational  move  will  be  one 
which  maximizes  the  total  payoff  to  all  of  the  players  involved.  This  leads  to  the  joint  move 
RN  above;  the  processor  which  will  benefit  the  most  from  the  use  of  the  FPA  requests  it. 
The  effect  of  this  is  to  minimise  the  total  amount  of  time  needed  for  the  two  processors  to 
complete  their  computations  (i.c.,  they  use  the  fewest  number  of  cycles). 

Let  us  now  allow  the  two  processors  to  do  a  little  more  meta-lcvcl  reasoning.  They 
realize  that  if  processor  1  does  not  request  the  FPA,  but  waits  until  the  other  processor 
is  done  with  it  before  starting  its  own  computation,  it  will  actually  complete  its  task  in  3 
units  of  time,  as  opposed  to  G.  The  second  processor  can  reason  similarly,  and  the  payoff 
matrix  should  therefore  be  replaced  with  this  one: 
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R 

N 

R 

4 

6 

3 

2 

N 

1 

4 

6 

3 

(14) 


Common  rationality  now  leads  to  the  joint  move  NR;  the  second  processor  requests  the 
FPA  while  the  first  waits  for  him  to  finish  using  it. 

Other  density  functions  can  be  used  to  produce  different  results.  If,  for  example,  it 
is  desired  to  have  the  slowest  processor  complete  its  task  in  the  shortest  time  possible, 
the  density  functions  need  merely  be  taken  to  be  very  sharply  increasing  functions  of  the 
time  needed  to  complete  the  task  without  access  to  the  FPA.  If  it  is  desired  to  have  the 
fastest  processor  complete  its  task  as  quickly  as  possible  (perhaps  to  begin  consideration 
of  another  problem),  the  density  functions  should  be  very  sharply  decreasing  functions  of 
the  time  needed  to  complete  the  task  with  access  to  the  FPA.  An  examination  of  (11)  will 
make  both  of  these  claims  clear.  Of  course,  all  of  these  methods  can  be  used  just  as  easily 
with  the  revised  payofT  function  (14)  as  with  the  earlier  version  (13). 

Let  me  conclude  by  describing  the  advantages  of  the  common  rationality  approach 
over  a  few  other  possibilities.  The  one  that  springs  to  mind  most  quickly  is  that  of  having 
the  task-assigning  machine  determine  which  of  the  subproccssors  should  use  the  FPA. 
This  will  be  reasonable  in  simple  situations,  but  there  may  well  be  a  substantial  amount 
of  analysis  involved  in  arriving  at  the  payoff  functions  appearing  in  (13),  and  this  analysis 
can  be  usefully  distributed  between  the  other  machines.  It  should  not  be  argued,  however, 
that  this  substantial  analysis  is  a  “waste  of  time” — Smith  has  pointed  out  [16]  that  meta¬ 
analysis  will  become  increasingly  important  .as  machines  tackle  more  ami  more  difficult 
problems. 

The  attractiveness  of  common  rationality  is  also  not  merely  a  consequence  of  the  harsh 
penalty  we  have  imposed  if  both  processors  request  access  to  the  FPA.  Suppose  that  the 
FPA  is  merely  assigned  at  random  in  such  a  case;  (13)  should  then  be  replaced  with: 


R 

N 

R 

2- 

4 

4 

2 

N 

1 

4 

6 

6 

Individual  rationality  still  forces  the  move  { RR };  although  this  move  is  no  longer  Parcto- 
suboptimal  (as  it  wjis  earlier),  there  may  nevertheless  be  many  situations  in  which  we  want 
to  avoid  it. 

A  final  possibility  would  be  to  have  the  processors  agree  on  a  payoff  function  and  to 
then  have  one  of  them  determine  what  action  should  be  taken.  The  point  of  theorem  6.2, 
however,  is  that  this  is  unnecessary:  the  various  processors  will  coordinate  their  actions 
automatically,  without  needing  to  select  a  single  machine  to  arbitrate  their  decisions. 
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Not  only  is  this  additional  complication  unnecessary,  but  there  are  many  situations 
where  it  will  be  outright  impossible — the  interaction  may  be  unexpected,  or  may  be  be¬ 
tween  agents  with  conflicting  goals.  There  are  a  variety  of  examples  in  [5],  although  the 
state  of  research  into  the  problems  of  interacting  intelligent  agents  seems  to  preclude  de¬ 
scribing  them  quantitatively  at  this  point. 

§9.  Conclusion 

The  decision  procedure  paradigm  is  a  new  one  in  which  to  examine  multi- agent  inter¬ 
actions:  the  results  of  this  paper  should  be  novel  to  both  game-theoretic  and  AI  audiences. 

The  advantage  of  this  paradigm  is  that  it  makes  it  possible  to  avoid  the  usual  assump¬ 
tion  that  the  decision  methods  of  the  various  agents  are  independent.  This  independence 
can  be  assumed,  but  it  is  only  one  of  a  variety  of  assumptions  which  can  be  made. 

One  of  these,  which  we  have  referred  to  as  common  rationality,  seems  especially  attrac¬ 
tive.  It  enables  us  to  understand  coordinated  action  in  terms  of  local  utility  maximization, 
and  solves  the  prisoner’s  dilemma.  We  have  also  showed  common  rationality  to  be  Pareto 
optimal  among  the  collection  of  all  possible  assumptions  which  might  be  made  by  inter¬ 
acting  agents  about  each  other. 

The  common  rationality  assumption  is  also  strong  enough  to  limit  the  communication 
needed  between  such  interacting  agents.  By  allowing  the  agents  to  communicate  infor¬ 
mation  describing  their  situation  instead  of  describing  their  proposed  intentions,  we  can 
establish  a  uniform  protocol  for  this  communication  while  avoiding  the  difficulties  that 
would  otherwise  be  a  consequence  of  potential  agent  dishonesty. 

The  difficulties  with  the  common  rationality  approach  appear  to  lie  in  two  assump¬ 
tions:  that  the  interacting  agents  have  matching  density  functions,  and  that  any  particular 
game  not  be  locally  or  globally  ambiguous.  Only  the  first  of  these  is  likely  to  present  a 
serious  difficulty  in  practice,  and  future  research  will  need  to  address  interaction  between 
agents  with  varying  expectations  about  the  situations  in  which  they  will  find  themselves. 
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This  research  has  been  supported  by  the  Office  of  Naval  Research  under  grant  number 
N00014-81-K-0004. 
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